Apparatus and method for tracking temporal variation of video content context using dynamically generated metadata

ABSTRACT

There are provided an apparatus and method for tracking temporal variation of a video content context using dynamically generated metadata, wherein the method includes generating static metadata on the basis of internal data held during an initial publication of video content and tagging the generated static metadata to the video content, collecting external data related to the video content generated after the video content is published, generating dynamic metadata related to the video content on the basis of the collected external data and tagging the generated dynamic metadata to the video content, repeating regeneration and tagging of the dynamic metadata with an elapse of time, tracking a change in content of the dynamic metadata, and generating and providing a trend analysis report corresponding to a result of tracking the change in the content.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to and the benefit of Korean PatentApplication No. 10-2017-0012415, filed on Jan. 26, 2017, the disclosureof which is incorporated herein by reference in its entirety.

BACKGROUND 1. Field of the Invention

The present disclosure relates to an apparatus and method for trackingtemporal variation of a video context by tracking metadata of videocontent that is dynamically generated in accordance with a temporalvariation feature.

2. Discussion of Related Art

Nowadays, high-quality content is flooding due to development of videoproduction technologies, personalization of high-performance videoproduction tools, and the like. This trend is expected to continue inthe future. Due to the change in the market, development of anapplication service for partially or entirely reusing previouslypublished video content is being attempted. For this, commercializationof technological development for automatic generation and tagging ofmetadata related to the published video content is required. Further,there is a need for the video content to utilize data generated afterthe publication of the content in addition to data initially publishedby the video content.

Various pieces of metadata may be tagged to an entire video file or apartial clip file of video content, and the corresponding content may besearched for on the basis of the pieces of metadata such that a contextof the video content may be utilized for republication of the content,development of a new application service, and the like. For this,research and development for automatically generating metadata relatedto video content are being carried out in terms of various aspects suchas sensor data collection, natural language processing, and imageprocessing.

In relation to the above, in Korean Patent Registration No. 10-0828166(Title of Invention: Method of extracting metadata from result of speechrecognition and character recognition in video, Method of searchingvideo using metadata and Record medium thereof) a moving picture isextracted/separated into frame units in which a screen transition occursto automatically generate and tag metadata of the moving picture. Also,a speaker's voice is recognized from a corresponding frame image, thevoice data is converted into text, and keywords are extracted from thetext data that results from the conversion. Also, a technology in whichcharacters are recognized from a corresponding frame image, thepreviously extracted keywords are searched for from the recognizedcharacters, weighting values are adjusted, metadata and a title areextracted from the keywords and the characters, and metadata based onvoice recognition and character recognition from the correspondingmoving picture is extracted on the basis of time information of a startshot and an end shot of an initially extracted frame is disclosed.

SUMMARY OF THE INVENTION

Embodiments of the present disclosure are directed to providing anapparatus and method for dynamically regenerating and tagging metadataaccording to temporal variation by continuously combining external datagenerated after publication of video content in addition to static datapublished during generating of the video content.

Further, embodiments of the present disclosure are directed to providingan apparatus and method for tracking context temporal variation of videocontent with time by tracking metadata of the video content dynamicallyregenerated and tagged in accordance with temporal variation.

However, objectives to be achieved by the embodiments of the presentdisclosure are not limited to the above, and the present disclosure mayhave other objectives.

To achieve the above-described objectives, according to an aspect of thepresent disclosure, an apparatus for tracking temporal variation of anvideo content context includes a first metadata generator/taggerconfigured to generate static metadata on the basis of internal dataheld during an initial publication of video content and tag thegenerated static metadata to the video content, an external datacollector configured to collect external data related to the videocontent generated after the video content is published, a secondmetadata generator/tagger configured to generate dynamic metadatarelated to the video content on the basis of the collected external dataand tag the generated dynamic metadata to the video content, and adynamic metadata-based video context temporal variation tracker/analyzerconfigured to repeat regeneration and tagging of the dynamic metadatawith an elapse of time, track a change in content of the dynamicmetadata, and generate and provide a trend analysis report.

According to another aspect of the present disclosure, a method fortracking temporal variation of an video content context includes (a)generating static metadata on the basis of internal data held during aninitial publication of video content and tagging the generated staticmetadata to the video content, (b) collecting external data related tothe video content generated after the video content is published, (c)generating dynamic metadata related to the video content on the basis ofthe collected external data and tagging the generated dynamic metadatato the video content, and (d) repeating regeneration and tagging of thedynamic metadata with an elapse of time, tracking a change in content ofthe dynamic metadata, and generating and providing a trend analysisreport.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features and advantages of the presentinvention will become more apparent to those of ordinary skill in theart by describing exemplary embodiments thereof in detail with referenceto the accompanying drawings, in which:

FIG. 1 is a block diagram of an apparatus for tracking temporalvariation of a context of video content according to an embodiment ofthe present disclosure;

FIG. 2 is a conceptual diagram for describing a process of repeatingregeneration and tagging of dynamic metadata related to video contentaccording to an embodiment of the present disclosure; and

FIG. 3 is a flowchart for describing a method for tracking temporalvariation of a context of video content according to an embodiment ofthe present disclosure.

FIG. 4 is a view illustrating an example of a computer system in which amethod according to an embodiment of the present invention is performed.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

Embodiments of the present disclosure will be described in detail belowwith reference to the accompanying drawings to allow one of ordinaryskill in the art to which the present disclosure pertains to easilypractice the present disclosure. However, the present disclosure may beimplemented in various other forms and is not limited to the embodimentsdescribed herein. To clearly describe the present disclosure, partsunrelated to the description have been omitted from the drawings, andlike parts are denoted by like reference numerals throughout thespecification. While the description is given with reference to theaccompanying drawings, different reference numerals may be given to thesame element according to the drawings. Reference numerals are merelyprovided for convenience of the description, and concepts, features,functions, or effects of each element is not limitedly interpreted bythe reference numerals.

Throughout the specification, when a certain part is described as“including” a certain element, this signifies that the certain part mayalso include another element rather than excluding the other elementunless particularly described otherwise, and this should not beunderstood as precluding the existence of or the possibility of addingone or more other features, numbers, steps, operations, elements, parts,or combinations thereof in advance.

In the present specification, “part” or “module” includes a unitrealized by hardware, a unit realized by software, and a unit realizedusing both hardware and software. A single unit may be realized usingtwo or more pieces of hardware, and two or more units may be realizeusing a piece of hardware. A “part” or “module” does not limitedly referto software or hardware and may be configured to be in an addressablestorage medium or configured to play one or more processors. Therefore,as an example, the “part” or “module” includes elements such as softwareelements, object-oriented software elements, class elements, and taskelements, processes, functions, attributes, procedures, subroutines, andsegments of program codes, drivers, firmware, micro-codes, circuits,data, databases, data structures, tables, arrays, and variables.Functions provided in such elements and “parts” (or “modules”) may becombined with a smaller number of elements and “parts” (or “modules”) ormay be further divided into additional elements and “parts” (or“modules”). Furthermore, elements and “parts” (or “modules”) may beimplemented to play one or more central processing units (CPUs) in adevice or a security multimedia card.

In an apparatus and method for tracking temporal variation of a contextof video content according to an embodiment of the present disclosurewhich will be described below, a context of video content is definedwith metadata tagged to a video, and changes in the context of the videocontent with time are tracked and analyzed. For this, in the apparatusand method for tracking temporal variation of a context of video contentaccording to an embodiment of the present disclosure, metadata relatedto a first time point t1 is generated by utilizing static text data suchas a script, a subtitle, and product placement information additionallygenerated when the video content is published, and when external datarelated to the video content is generated after the video content ispublished, the external data is additionally collected, and metadata isdynamically regenerated and tagged.

Hereinafter, the apparatus and method for tracking temporal variation ofa context of video content according to an embodiment of the presentdisclosure will be described in detail with reference to the drawings.

FIG. 1 is a block diagram of an apparatus for tracking temporalvariation of a context of video content according to an embodiment ofthe present disclosure.

FIG. 2 is a conceptual diagram for describing a process of repeatingregeneration and tagging of dynamic metadata related to video contentaccording to an embodiment of the present disclosure.

As illustrated in FIG. 1, an apparatus for tracking temporal variationof a video content context 100 includes a first metadatagenerator/tagger 110, an internal data storage 120, an external datacollector 130, an external data storage 140, a second metadatagenerator/tagger 150, and a dynamic metadata-based video contexttemporal variation tracker/analyzer 160.

The first metadata generator/tagger 110 generates metadata (hereinafterreferred to as “first metadata”) on the basis of static text datapublished in generating video content and tags the generated firstmetadata to the corresponding video content.

A unit of video content for which the first metadata is generated andtagged includes a unit such as an entire episode or a partial, clippedvideo from the episode, but embodiments are not limited thereto.

Here, the first metadata generator/tagger 110 generates the firstmetadata on the basis of static text data published in generating videocontent including planning, shooting, and editing the video content(that is, before screening or broadcasting the video content).

The first metadata generator/tagger 110 may collect and utilize thestatic text data published in the generating of the video content andgenerate topics and keywords of the video content on the basis of thetext data directly utilized in producing the video content. For example,the first metadata generator/tagger 110 may extract topics and keywordsby learning a topic model on the basis of the collected text data.

The first metadata generator/tagger 110 acquires static text datapublished in the generating of the video content from the internal datastorage 120.

Static text data preregistered for each of a plurality of pieces ofvideo content is stored in the internal data storage 120, and the statictext data may include data of original work, a storyboard, a script, asubtitle, product placement information, and the like generated in theplanning, shooting, and editing of the video content.

Even after the video content is published to a user in the form in whichthe video content is screened or broadcasted, external data may begenerated in various formats. For example, after video content such as adrama or documentary is broadcasted (that is, published), external textdata is continuously generated in the form in which pieces of contentrelated to various subject matters such as a historical background orknowledge related to a topic of the corresponding video content, ashooting place, current statuses of actors, and items worn by the actorsare mentioned in blogs, social network service (SNS), and personal mediachannels. The apparatus for tracking temporal variation of a videocontent context 100 according to an embodiment of the present disclosureutilizes such external text data in analyzing trends of a context of avideo.

The external data collector 130 collects external text data generatedafter the video content is published and stores the collected externaltext data in the external data storage 140. Also, the external datacollector 130 may search for the external text data by inputting topicsand keywords of static metadata and pre-generated dynamic metadata.

Here, the external data collector 130 collects external text dataincluding time information at which text data is published in relationto any piece of video content. For example, the external data collector130 continuously collects and stores external text data by searchingmedia such as news, blogs, SNS, personal media channels online in realtime or periodically using keywords.

The second metadata generator/tagger 150 regenerates dynamic metadata(hereinafter referred to as “second metadata”) related to any piece ofvideo content on the basis of the external text data collected by theexternal data collector 130 and tags the regenerated second metadata tothe corresponding video content.

The second metadata generator/tagger 150 expands and updates topics andkeywords of the static metadata using the collected external data andpublication time information of the external data, and tags the secondmetadata, along with publication time information of the secondmetadata, to the video content.

For example, the second metadata generator/tagger 150 adds the externaltext data to the topic model generated on the basis of the pre-collectedstatic text data and periodically/non-periodically re-learns the topicmodel. In the re-learning of the topic model as above, the existingtopic model is updated, and new second metadata to which the latesttrends are reflected is generated in addition to the first metadatacontained in the video content.

The second metadata generator/tagger 150 combines newly-collectedexternal data and generation time information of the data with a resultlearned with the topic model that is modeled during generation of thefirst metadata to regenerate the second metadata. Here, the dynamicmetadata-based video context temporal variation tracker/analyzer 160repeats regeneration/tagging of the second metadata using theperiodically collected external text data at a time point at which thevideo content is being continuously screened or broadcasted and, whenthe screening or broadcasting of the video content is completed,regenerates/tags the second metadata using the non-periodicallycollected external text data.

Referring to FIG. 2, a process in which the dynamic metadata (that is,the second metadata) is regenerated and tagged is repeatedly performedafter the static metadata (that is, the first metadata) is generated andtagged.

For example, metadata m¹_t¹ is generated on the basis of static datasuch as a script, a subtitle, and product placement informationgenerated at a time t1 at which video content is initially published. Atopic model is learned on the basis of the static data collected asabove, and topics and keywords are extracted.

Also, after the video content is published, external data (that is,dynamic data) such as SNS, blogs, news related to content of the videocontent is continuously generated. The external data generated in thisway is continuously additionally collected on the basis of the metadatam¹_t¹ for the first to nth times, and the collected external data isstored in the external data storage. Here, the stored external data isrepeatedly used in a process in which the dynamic metadata isregenerated and tagged periodically or non-periodically. A unit of videocontent to which metadata is tagged includes a unit such as an entireepisode or a partial, clip unit from the episode. A method forextracting a clip unit from an episode may be an automatic method, asemi-automatic method, or a manual method, but embodiments are notlimited thereto.

Metadata is dynamically regenerated by utilizing the external datagenerated after the video content is published by utilizing the topicmodel based on the static data generated at the time point at which thevideo content is published, and the regenerated dynamic metadataincludes temporal information. In this way, the dynamic metadataincluding temporal information may be generated by utilizing variouspieces of external data across topics of the video content after thevideo content is published and re-learning the topic model learned onthe basis of internal factors of the video content when the videocontent is initially programmed and broadcasted or screened. Here, howmetadata representing a piece of video content is limited to a specifictopic and then changes in accordance with a temporal variation featuremay be tracked and content thereof may be analyzed.

The dynamic metadata-based video context temporal variationtracker/analyzer 160 defines a video context as metadata representingvideo content and tracks how the video context temporally changes whenthe dynamic metadata (that is, the second metadata) is regenerated.

For example, the dynamic metadata-based video context temporal variationtracker/analyzer 160 may analyze content of metadata contradictory toexisting metadata resulting from reinterpretation of video content suchas a drama or documentary that deals with a specific topic in accordancewith social issues, viewers' opinions, and current statuses of actorswith time and write a trend analysis report by tracking how an existingvideo context is temporally changing.

The first metadata and the second metadata described above withreference to FIGS. 1 and 2 may be stored and managed in a metadatastorage (not illustrated), which is a separate storage space.

Hereinafter, a method for tracking temporal variation of a context ofvideo content according to an embodiment of the present disclosure willbe described with reference to FIG. 3.

FIG. 3 is a flowchart for describing a method for tracking temporalvariation of a context of video content according to an embodiment ofthe present disclosure.

As illustrated in FIG. 3, when video content is initially publishedfirst (S310), static data set in relation to the video content ischecked (S320).

Then, static metadata based on the static data of the video content isgenerated and tagged to the video content (S330).

Then, external text data is repeatedly collected on the basis of thestatic metadata with time after the video content is published (S340).

Then, dynamic metadata is regenerated on the basis of the external textdata at every time point at which the external text data is collected,and the regenerated dynamic metadata is tagged to the video content(S350).

Then, the regenerated dynamic metadata is periodically ornon-periodically tracked in accordance with temporal variation (S360).

Then, temporal variation of a video context is tracked on the basis of aresult of tracking the dynamic metadata in accordance with the temporalvariation, a result thereof is analyzed, and a trend analysis report isgenerated and provided (S370).

An intelligent marketing analysis method using an intelligent marketinganalysis system by processing unstructured big data according to anembodiment of the present disclosure described above may be implementedin the form of a computer program stored in a medium executed by acomputer or a recording medium including computer-executableinstructions. Such a recording medium may be a computer-readable mediumthat includes any computer-accessible available media, volatile andnonvolatile media, and removable and non-removable media. Thecomputer-readable medium may also include a computer storage medium, andthe computer storage medium includes volatile and nonvolatile media andremovable and non-removable medium implemented with any method ortechnology for storing information such as a computer-readableinstruction, a data structure, a program module, and other data.

Although the method and system of the present disclosure have beendescribed in relation to particular embodiments, elements or operationsthereof may be partially or entirely implemented using a computer systemhaving a universal hardware architecture.

The description of the present disclosure given above is merelyillustrative, and one of ordinary skill in the art to which the presentdisclosure pertains should understand that the present disclosure may beeasily modified in other specific forms without changing the technicalspirit or essential features of the present disclosure. Therefore, theembodiments described above are merely illustrative in all aspects andshould not be understood as limiting. For example, each elementdescribed as a single element may be distributed into a plurality ofelements, and likewise, elements described as being distributed may becombined into a single element.

According to any one aspect of the present disclosure, because topicmodel-based metadata used to search for a meaning that video contentimplies changes with time, external data continuously generated afterthe video content is published is utilized in a topic model in additionto static text data generated in an initial publication of the videocontent.

In this way, by using metadata of video content that is dynamicallyregenerated and tagged in accordance with temporal variation of acontext of the video content, new metadata related to video contentpublished before a predetermined amount of time can be generated inaccordance with new issues and trends, and the corresponding videocontent can be searched for or utilized in accordance with the latestissues.

The method according to an embodiment of the present invention may beimplemented in a computer system or may be recorded in a recordingmedium. FIG. 4 illustrates a simple embodiment of a computer system. Asillustrated, the computer system may include one or more processors 421,a memory 423, a user input device 426, a data communication bus 422, auser output device 427, a storage 428, and the like. These componentsperform data communication through the data communication bus 422.

Also, the computer system may further include a network interface 429coupled to a network. The processor 421 may be a central processing unit(CPU) or a semiconductor device that processes a command stored in thememory 423 and/or the storage 428.

The memory 423 and the storage 428 may include various types of volatileor non-volatile storage mediums. For example, the memory 423 may includea ROM 424 and a RAM 425.

Thus, the method according to an embodiment of the present invention maybe implemented as a method that can be executable in the computersystem. When the method according to an embodiment of the presentinvention is performed in the computer system, computer-readablecommands may perform the producing method according to the presentinvention.

The method according to the present invention may also be embodied ascomputer-readable codes on a computer-readable recording medium. Thecomputer-readable recording medium is any data storage device that maystore data which may be thereafter read by a computer system. Examplesof the computer-readable recording medium include read-only memory(ROM), random access memory (RAM), CD-ROMs, magnetic tapes, floppydisks, and optical data storage devices. The computer-readable recordingmedium may also be distributed over network coupled computer systems sothat the computer-readable code may be stored and executed in adistributed fashion.

The scope of the present disclosure is defined by the claims belowrather than the detailed description above, and all changes ormodifications derived from the meaning and the scope of the claims andtheir equivalents should be interpreted as belonging to the scope of thepresent disclosure.

What is claimed is:
 1. An apparatus for tracking temporal variation of avideo content context, the apparatus comprising: a processor configuredto: generate static metadata based on internal data held during aninitial publication of video content, tag the generated static metadatato the video content, collect external data related to the video contentgenerated after the video content is published, generate dynamicmetadata related to the video content based on the collected externaldata, tag the generated dynamic metadata to the video content,regenerate the dynamic metadata with an elapse of time, tag theregenerated dynamic metadata to the video content, track a temporalchange in context of the regenerated dynamic metadata from the generateddynamic metadata, and generate and provide a trend analysis report of acontext of the video based on the tracked temporal change in the contextof the regenerated dynamic metadata, wherein the processor comprises: afirst metadata generator/tagger configured to generate the staticmetadata based on the internal data held during the initial publicationof the video content, and tag the generated static metadata to the videocontent, an external data collector configured to collect the externaldata related to the video content generated after the video content ispublished, a second metadata generator/tagger configured to generatedynamic metadata related to the video content based on the collectedexternal data, and tag the generated dynamic metadata to the videocontent, and a dynamic metadata-based video context temporal variationtracker/analyzer configured to regenerate the dynamic metadata with theelapse of time, tag the regenerated dynamic metadata to the videocontent, track the temporal change in context of the regenerated dynamicmetadata from the generated dynamic metadata, and generate and providethe trend analysis report of the context of the video based on thetracked temporal change in the context of the regenerated dynamicmetadata, wherein the second metadata generator/tagger is furtherconfigured to expand and update topics and keywords of the staticmetadata using the collected external data and publication timeinformation of the collected external data, and tag other metadata,along with publication time information of the other metadata, to thevideo content.
 2. The apparatus of claim 1, wherein the first metadatagenerator/tagger is further configured to generate the static metadataby extracting topics and keywords based on static text data includingany one or any combination of any two or more of a script, a subtitle,product placement information, and a storyboard pre-generated at a timepoint at which the video content is initially published.
 3. Theapparatus of claim 1, wherein the external data collector is furtherconfigured to search for the external text data by inputting topics andkeywords of the static metadata and pre-generated dynamic metadata, andstore the collected external text data in an external data storage. 4.The apparatus of claim 1, wherein the dynamic metadata-based videocontext temporal variation tracker/analyzer is further configured toregenerate the dynamic metadata, tag the regenerated dynamic metadata byusing periodically collected external text data at a time point at whichthe video content is continuously screened or broadcasted and, inresponse to the screening or broadcasting of the video content beingcompleted, regenerate and tag the dynamic metadata usingnon-periodically collected external text data.
 5. Aprocessor-implemented method for tracking temporal variation of a videocontent context using dynamic metadata, the method comprising:generating static metadata based on internal data held during an initialpublication of video content; tagging the generated static metadata tothe video content; collecting external data related to the video contentgenerated after the video content is published; generating dynamicmetadata related to the video content based on the collected externaldata; tagging the generated dynamic metadata to the video content; andregenerating the dynamic metadata with an elapse of time; tagging theregenerated dynamic metadata to the video content; tracking a temporalchange in context of the regenerated dynamic metadata from the generateddynamic metadata; and generating and providing a trend analysis reportof a context of the video based on the tracked temporal change in thecontext of the regenerated dynamic metadata, wherein the generating ofthe dynamic metadata includes expanding and updating topics and keywordsof the static metadata using the collected external data and publicationtime information of the collected external data, and tagging othermetadata, along with publication time information of the other metadata,to the video content.
 6. The method of claim 5, wherein the generatingof the static metadata and the tagging of the generated static metadatainclude generating the static metadata by extracting topics and keywordsbased on static text data including any one or any combination of anytwo or more of a script, a subtitle, product placement information, anda storyboard pre-generated at a time point at which the video content isinitially published.
 7. The method of claim 5, wherein the collecting ofthe external data includes searching for the external text data byinputting the topics and the keywords of the static metadata andpre-generated dynamic metadata.
 8. The method of claim 5, wherein theregenerating of the dynamic metadata includes repeating regeneration andtagging of the dynamic metadata using periodically collected externaltext data at a time point at which the video content is continuouslyscreened or broadcasted and, in response to the screening orbroadcasting of the video content being completed, regenerating andtagging the dynamic metadata using non-periodically collected externaltext data.
 9. The method of claim 5, wherein the static metadata isgenerated using topic modeling in machine learning.
 10. The method ofclaim 5, wherein the dynamic metadata is regenerated through topicextension by collecting the external data.